Archaeologists have traditionally collected artifacts during survey in order to analyze them in a laboratory setting and curate the artifacts and associated documentation for future analysis, interpretation, and preservation. In recent decades, however, there has been a trend in the western United States to avoid collection during survey and to relegate most artifact analysis to the field, typically conducted by field crew. Despite heavy reliance on in-field analysis to characterize sites during survey, very little is known about how accurate and adequate in-field analysis is for site interpretation and management. This article presents the findings of a pilot experiment that tested in-field analysis and digital photograph analysis at two sites in the western United States using multiple quantitative measures and qualitative assessments. The results of the analysis show that in-field analysis has a strong potential to yield inaccurate and highly variable results that can lead to the misidentification and misinterpretation of important site components. We recommend that sample collections be made during survey, where possible, and properly curated. For projects in which in-field analysis is used, we recommend that in-field analysis be tested using similar methods to determine whether it can meet an acceptable standard.